The invention relates to the virtual endoscopy, and in particular to determining a viewpoint for navigating a virtual camera through a biological object with a lumen.
Traditional endoscopy involves the use of an endoscope which is inserted into a patient to allow direct visual inspection of, for example, the colon. The technique is relatively invasive and uncomfortable, and often requires heavy patient sedation. Accordingly, virtual endoscopy, which is based on an analysis of medical image data (computer simulated endoscopy), is often preferred.
Patient medical imaging methods, such as computer-assisted tomography (CT), magnetic resonance imaging (MRI), ultrasound and positron-emission tomography (PET), generate large three-dimensional (3D) volume data sets representing all or part of a patient's body. These volume data sets are highly detailed and allow complex studies of the body to be made. Virtual endoscopy is a technique in which a virtual camera is made to travel along a biological object with a lumen (such as a colon, blood vessel or bronchial tree), with the image of the lumen seen by the virtual camera presented to a user for diagnostic and other purposes. Virtual endoscopy is less invasive for the patient and provides the clinician with a much greater degree of flexibility in the way he can view the lumen of interest. For example, the clinician can choose to observe the virtual lumen from almost any direction and on any scale.
A common analysis technique for medical image data is therefore to render a series of two-dimensional images from the viewpoint of a camera travelling along a path within a lumen of interest. The series of images may be displayed to a clinician in sequence to provide a virtual “fly-through” of the lumen of interest. To navigate the viewpoint through the lumen, a path for the virtual camera to follow is usually determined in advance because even experienced clinicians can have difficulty in manually controlling the virtual camera movement in real time.
There are a number of well-known methods for calculating suitable paths for the camera to follow. These paths are frequently referred to as centerlines because they are usually designed to follow as closely as possible a central route through the lumen. Methods for calculating suitable paths include those based on mathematical techniques designed for wall avoidance [1, 2], those based on mathematical techniques that use erosion to determine a centerline along a lumen [3], those based on labeling points in the data set with their distance from an end of the path using a wavefront model to avoid obstacles and calculating the path by moving from point to point according to the closest distance to the end point [4], those based on obtaining an initial path by using a distance label map to connect start and end voxels in the data set via intermediate voxels according to their distance from the start and end voxels, and then centering the path using maximum diameter spheres [5, 6], and those based on determining navigation steps by using ray casting to find the longest ray from a camera position to the wall and weighting this with the existing viewing direction of the camera to obtain a new direction for movement of the camera, and repeating this for each step [7]. General data collection and processing methods for viewing three-dimensional patient images have also been described [8, 9]. A number of commercial virtual endoscopy systems and the various approaches taken to camera navigation and to the image processing necessary to represent a realistic view of the lumen are discussed by Bartz [10].
Once the locus of the flight path (centerline) is determined, a number of locations along the path are selected as locations from which to render images that will comprise frames in the virtual flythrough of the lumen. Thus the flythrough comprises images rendered from locations corresponding to incremental steps along the flight path. For a colon, the locations for generating images will typically be evenly spaced along the flight path with separations (i.e. step sizes) on the order of 1 mm or so.
In addition to selecting locations for the virtual camera position, it is also necessary to select a view direction (i.e. orientation) for the camera to render the images. Conventionally the camera will be oriented so that it points towards a location on the flight path a fixed distance ahead of its current position. Thus, at the start location the virtual camera will be oriented so that its view vector is directed towards a position on the flight path that the camera will reach in, say, four steps time. The orientation of the camera about its view vector (roll angle) at the initial location will typically be selected arbitrarily. Once an image has been rendered for the initial location and camera orientation, the virtual camera undergoes a translation movement to take it to the next location on the flight path. After the translation, the camera orientation will be such that it will not be pointing along the desired view direction for the new location (at least in the general case). Accordingly, the camera is re-oriented so that it again points towards a position which is the selected fixed distance ahead of the camera's present location on the flight path (e.g. four steps ahead in this example). This can be achieved using conventional vector algebra, and, depending on the shape of the flight path, the camera rotation may include components about any of the camera's pitch, roll or yaw axes. An image may then be rendered which corresponds to the camera's new location and view direction. The camera then moves to the next location along the flight path, where the process is repeated. This continues until the camera completes the fly through (i.e. reaches the last location on the flight path).
A problem with this approach is that the flythrough view can become disorientating for a viewer as the virtual camera rolls, pitches and yaws along what is typically a complex path of loops, twists and turns. This can hinder the interpretation of the data and in some cases can even cause nausea for the clinician viewing the data.
A further problem arises when a clinician wishes to compare different data sets representing the same lumen. There are a number of reasons why a clinician may want to do this. For example, medical image data will typically be obtained with the patient in a range of orientations, e.g., on his back (supine), on his front (prone), and/or on his side(s) (lateral decubitus). A clinician will usually want to compare flythroughs of all available data sets for these orientations to allow a thorough examination of the lumen. In other cases there may be multiple data sets representing the lumen of interest which have been obtained at different times. The clinician will again usually want to directly compare flythroughs of these data sets, e.g., to look for temporal changes in the lumen.
However, with the above described technique for orienting a virtual camera, the camera will adopt an effectively random roll angle about its view vector at locations along the flight path. This means a feature of interest might appear in the top left of a rendered image in a flythrough of one data set, but in the bottom right of a rendered image in a flythrough of another data set, even if the data sets represent the same lumen. This makes it difficult to register and reconcile potential features of interest in one data set with those in another.
This lack of registration between images from different flythroughs increases the difficulty in interpreting the data. Furthermore, it leads to an increased risk that the presence of the same feature in multiple data sets will not be noticed because it appears in different locations in different flythroughs. This can be problematic because features seen in one data set but missed in another will frequently be discounted as not being of interest. In virtual colonoscopy, for example, a feature seen in one data set, but not another, will usually be ignored as being residual fecal matter that has moved between data sets. However, the feature may in fact be a permanent polyp that has been overlooked in one data set, or seen but not recognized as the same feature.
Attempts to address these problems are described by Huang et al. [11]. In one technique, Huang et al., disclose defining a preferred up direction for a virtual camera. During a flythrough, the camera is then oriented about its view direction to align its preferred up direction as far as possible with an up direction in a world coordinate system. However, as Huang et al. explain, this approach still leads to sudden twists and turns which can disorient a user and induce nausea. Huang et al. note that Paik et al. [12] attempt to address this by linking the orientation to one frame to that in a previous frame. However, this again leads to camera orientations which are not reproducible in different data sets. Huang et al. propose an alternative scheme in which a virtual camera is oriented about its roll axis to align with teniae coli structures of the colon. However, a drawback with this approach is that it cannot be performed automatically since a significant amount of clinician input is required to manually identify the teniae coli in the first instance. Automation is generally preferred for medical image data pre-processing since this saves clinician time, both in terms of the clinician having to take part in the pre-processing, and also in terms of the clinician having to wait for the results of the pre-processing before continuing his analysis. Automation can also help improve objectivity and reproducibility in the analysis because a possible source of clinician subjectivity is removed.
Accordingly, there is a need for an improved method of automatically determining a viewpoint orientation for navigating a virtual camera through a biological object with a lumen.